'No Effect' Conclusions in Studies Reporting Nonsignificant Results Are Potentially Incorrect.

PURPOSE: To examine the spectrum of effect sizes in line with "no effect" claims in clinical studies published in high-impact orthopaedic journals. METHODS: Eight orthopaedic journals with the highest impact factors for 2018 were included in this study. The titles and abstracts of all journal articles published in these journals in 2019 were screened for no-effect statements. The effect sizes for effects claimed to be no effect were calculated. The asymmetry of 95% confidence interval (CI) tails of effect estimated in relation to 0 was examined by calculating the ratio between the tail lengths. RESULTS: The analysis included 255 articles reporting coefficients sufficiently. The median Cohen's d value was 0.15 (interquartile range [IQR] 0.06 to 0.25) in the studies comparing means and the median ϕ value in the studies comparing frequency distributions was 0.06 (IQR 0.04 to 0.15). In studies reporting odds ratios or hazard ratios, the median estimate value was 1.16 (IQR 1.09 to 1.34). The median asymmetry ratio for all studies was 1.9 (IQR 1.3 to 4.0). Asymmetry ratio values exceeded 5 in 55 studies (22%). CONCLUSIONS: No-effect statements are used for a wide variety of observed effects. Despite the conclusion of no effect, observed effects advocated toward actual effects. In addition, many cases in which the asymmetry of CIs related to the 0-effect estimate showed a high tendency of effect direction. CLINICAL RELEVANCE: Rather than emphasizing dichotomized interpretation of statistical inference, reporting observed coefficients with elaboration of related uncertainty and compatibility with meaningful effect sizes in the specific context is encouraged.

Long-term subjective results and radiologic prognosis of a distal radius fracture in working-aged patients - a prognostic cohort study of 201 patients.

OBJECTIVE: To investigate long-term outcomes associated with distal radius fracture (DRF) in working-aged patients. The authors hypothesized that the majority of patients experience no permanent loss of function when measured with patient-rated wrist evaluation (PRWE). METHODS: This was a retrospective cohort study of patients with a DRF aged between 18 and 65 years. The primary outcome measure was PRWE score at a minimum of 4 years after DRF. Secondary outcome measures were pain catastrophizing scale (PCS) and radiographic measurements. RESULTS: Of 201 patients included, 179 were primarily treated non-operatively with a 5-week cast treatment and 22 were primarily operated. The mean follow-up duration was 5 years. The mean PRWE score was 10.9 (95% confidence interval 8.4, 13.4) and median PRWE was 3.5 (interquartile range, 0.0-13.0). There was minor correlation between PCS and PRWE score (correlation coefficient [CC] 0.3), and between PRWE score and dorsal angulation of the fracture measured after closed reduction (CC 0.2) and in one-week follow-up radiographs (CC 0.2). CONCLUSIONS: Working-aged patients seem to gain nearly normal wrist function after DRF in longer follow-up. Pain catastrophizing appears to correlate with long-term treatment outcome.

Estimates of the mean difference in orthopaedic randomized trials: obligatory yet obscure.

BACKGROUND: Randomized controlled trials in orthopaedics are powered to mainly find large effect sizes. A possible discrepancy between the estimated and the real mean difference is a challenge for statistical inference based on p-values. We explored the justifications of the mean difference estimates used in power calculations. The assessment of distribution of observations in the primary outcome and the possibility of ceiling effects were also assessed. METHODS: Systematic review of the randomized controlled trials with power calculations in eight clinical orthopaedic journals published between 2016 and 2019. Trials with one continuous primary outcome and 1:1 allocation were eligible. Rationales and references for the mean difference estimate were recorded from the Methods sections. The possibility of ceiling effect was addressed by the assessment of the weighted mean and standard deviation of the primary outcome and its elaboration in the Discussion section of each RCT where available. RESULTS: 264 trials were included in this study. Of these, 108 (41 %) trials provided some rationale or reference for the mean difference estimate. The most common rationales or references for the estimate of mean difference were minimal clinical important difference (16 %), observational studies on the same subject (8 %) and the 'clinical relevance' of the authors (6 %). In a third of the trials, the weighted mean plus 1 standard deviation of the primary outcome reached over the best value in the patient-reported outcome measure scale, indicating the possibility of ceiling effect in the outcome. CONCLUSIONS: The chosen mean difference estimates in power calculations are rarely properly justified in orthopaedic trials. In general, trials with a patient-reported outcome measure as the primary outcome do not assess or report the possibility of the ceiling effect in the primary outcome or elaborate further in the Discussion section.

Statements considering intervention effects in Finnish clinical practice guidelines: Recommending interventions with non-numeric effect-sizes or unspecified outcomes.

RATIONALE, AIMS AND OBJECTIVES: Representation of benefits and harms associated with specific interventions in an understandable and comparable way is crucial for informed decision making that clinical practice guidelines (CPGs) aim to enhance. Therefore, we investigated how statements concerning the effects of interventions considered and described benefits and harms, magnitude of effect and its uncertainty, numeric and non-numeric information, and outcomes in Finnish CPGs. METHODS: We selected 10 CPGs on common diseases and risk factors published by The Finnish Medical Society, Duodecim. All the statements which were graded with the level of evidence from high to very low (levels A-D) were included in analyses. From these statements, assessments were made regarding whether the statement considered benefits or harms, whether relative or absolute numeric measures were shown, whether the statement supported or was against the intervention considered, and what outcome was reported. RESULTS: Of the 10 CPGs, 448 statements were assessed. Most of the statements of effects considered intervention benefits (87%) rather than harms. Half of the statements considering harms were represented in a way that supported the intervention. Most of the statements (94%) did not include numeric estimates of magnitude of the effect. When numeric estimates of magnitude of the effect were present, they were most frequently relative measures and were typically placed in a statement considering (a) intervention benefits with a primary outcome, (b) given the grade of A for level of evidence, and (c) that supported the use of intervention. CONCLUSIONS: In the Finnish CPGs, the statements were rarely framed with both absolute and relative numeric measures of an intervention's effect. Harms were rarely reported with a grade indicating the level of evidence. The users of CPGs would benefit from more consistent and understandable framing of statements considering both benefits and harms of interventions.

Verbal communication of effect-sizes, possible comparators, and uncertainty of evidence in the Finnish clinical practice guidelines: Omitting effect-sizes and comparators without expressing much uncertainty.

RATIONALE, AIMS, AND OBJECTIVES: Our aim was to investigate verbal representations of intervention effect-size, uncertainty of evidence, and possible intervention comparators in statements concerning effects of interventions in Finnish clinical practice guidelines. METHOD: We selected 10 clinical practice guidelines on common diseases and risk factors published by The Finnish Medical Society Duodecim. All the statements that considered beneficial effects of interventions and that were graded with a level of evidence (levels A, high, to D, very low) were included in analyses. We assessed whether the statements verbally represented intervention effect-size, uncertainty of evidence or a possible comparator, and the reported outcome. RESULTS: Of 385 statements, verbal representation of beneficial effect-sizes occurred in 25 (6%) statements. Most (72%) statements indicated that intervention had a beneficial effect, but did not specify effect-size. Less than half (42%) of the statements represented uncertainty verbally. Comparisons to placebo or no-treatment were rare (3%) and 18% of the statements compared interventions to other treatments. Against instructions, a considerable part (35%) of statements with B-level evidence did not represent uncertainty. CONCLUSION: Communicating beneficial intervention effects, effect-sizes, possible comparators, and uncertainty of evidence require much broader attention in the clinical practice guideline context.

Distal radius fractures in the elderly population.

We found no clear evidence of the clinical superiority of distal radius fracture surgery among older adults at one year.Surgical treatment, however, may yield a faster recovery to previous level of activity in elderly patients.With operative treatment, hardware-based problems may warrant secondary operations and implant removal, whereas in non-operative treatment, symptomatic loss of alignment and malunion can occur.In elderly patients, non-operative treatment can be considered to be the gold standard. Cite this article: EFORT Open Rev 2020;5:361-370. DOI: 10.1302/2058-5241.5.190060.

Cost-Effectiveness of Operative Versus Non-Operative Treatment for Clavicle Fracture: a Systematic Literature Review.

PURPOSE OF REVIEW: Operative and non-operative treatment of midshaft clavicle fractures seems to yield comparative functional results. Furthermore, it has been suggested that surgery is more expensive compared with non-operative treatment of clavicle fracture. Cost-effectiveness seems to be more important in trends of treatment decisions. The purpose of this study is to investigate the cost-effectiveness of clavicle fracture treatment. RECENT FINDINGS: Seven publications were selected, and 5 studies showed that operative treatment is more expensive than non-operative treatment. The mean overall cost per person in discounted prices was 10,230 USD for operative and 7923 USD for non-operative treatment. The mean absence from work ranged 8-193 and 24-69 days for operative and non-operative treatment, respectively. Studies varied in methods of assessing the cost-effectiveness of treatment modalities. Based on this literature review, routine operative treatment seems to be more expensive. In some cases, operative treatment might be more cost-effective. In all studies, direct and indirect costs of health care were calculated, but a great heterogeneity exists in the sources of cost data between countries. The cost-effectiveness of the treatment of clavicle fracture depends strongly on the cost of operative treatment and length of absence from work. Cost-effectiveness analysis could be a routine in RCT studies in the future.

Two casting methods compared in patients with Colles' fracture: A pragmatic, randomized controlled trial.

BACKGROUND: Distal radius fractures are common fractures and the cornerstone of treatment remains immobilization of the wrist in a cast. At present, there is a scarcity of studies that compare different cast immobilization methods. The objective of the study was therefore to compare volar-flexion and ulnar deviation cast to functional cast position in the treatment of dorsally displaced distal radius fracture among elderly patients. METHODS AND FINDINGS: We performed a pragmatic, randomized, controlled trial in three emergency centers in Finland. After closed reduction of the fracture, the wrist was placed in either volar-flexion and ulnar deviation cast or functional cast position. The follow-up was 12 months. The primary outcome was patient-rated wrist evaluation (PRWE) score at 12 months. The secondary outcomes were Quick-DASH score, grip strength, health-related quality of life (15D), and pain catastrophizing scale. The number of complications was also recorded. In total, 105 participants were included in the study. Of these, 88% were female and the mean age was 73.5 (range 65-94) years. In the primary analysis, the mean difference in patient-rated wrist evaluation measure between groups was -4.9 (95% CI: -13.1.- 3.4., p = .24) in favor of the functional cast position. Operative treatment due to loss of reduction of fracture was performed for four patients (8%) in the FC group and for seven patients (13%) in the volar-flexion and ulnar deviation cast group (OR: 0.63, 95% CI: 0.16-2.1). CONCLUSION: In this study, the data were consistent with a wide range of treatment effects when comparing two different cast positions in the treatment of distal radius fracture among elderly patients at 12-month follow-up. However, the functional cast is more likely to be superior when compared to volar-flexion and ulnar deviation cast. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT02894983 Accessible: https://clinicaltrials.gov/ct2/show/NCT02894983.

Assessing variability and uncertainty in orthopedic randomized controlled trials.

Background and purpose - Low statistical power remains endemic in clinical medicine including orthopedics and manifests as high uncertainty and wide confidence intervals (CI). We evaluated the reporting and correspondence between power calculation and observed data on key parameters of variability and uncertainty in orthopedic randomized controlled trials (RCTs).Material and methods - RCTs with 1:1 allocation published in 8 major orthopedic journals between 2016 and 2017 with one continuous primary outcome were included in the review. The components of power calculation and observed standard deviation (SD), mean difference (MD), and confidence interval (CI) of MD between groups were assessed for primary outcome.Results - 160 RCTs were included, of which 93 (58%) and 138 (86%) studies reported the estimated SD and MD in the power calculation, respectively. The median ratio of the estimated SD and SDs observed in the data was 1.0 (IQR -0.76 to 1.32) for 69 (43%) studies. Only 31 of 138 studies reported the CI of MD in primary outcome. In 42% of the negative studies, the estimated MD was included in the CI of the observed MD.Interpretation - The key parameters of data variability, both in power analyses and in final study results, were poorly reported. Low power in orthopedics may result from too high an estimated effect size due to an overoptimistic estimate of MD between study groups. In almost half of the studies, overlap of the CI of the observed MD and estimated MD suggested that the reported results of these studies were inconclusive.

Revisiting the Sample Size and Statistical Power of Randomized Controlled Trials in Orthopaedics After 2 Decades.

BACKGROUND: A study published in 2001 reported that sample sizes in the randomized controlled trials (RCTs) published in major orthopaedic journals in 1997 were too small, resulting in low power to detect reasonable effect sizes. Low power is the fundamental reason for the poor reproducibility of research findings and serves to erode a cornerstone of the scientific method. The aim of this study was to ascertain whether improvements have been made in orthopaedic research during the past 2 decades. METHODS: The electronic table of contents from the 2016 and 2017 volumes of 7 major orthopaedic journals were searched issue by issue in chronological order to identify possible RCTs. A posteriori (after-the-fact) power to detect small, medium, and large effect sizes, defined by the Cohen d value, were calculated from the sample sizes reported in the studies. The power to detect effect sizes associated with the most commonly used patient-reported outcome measures (PROMs) was also calculated. Finally, the use of a priori power analysis in the included studies was assessed. RESULTS: In total, 233 studies were included in the final analyses. None of the negative studies had sufficient power (≥0.80) to detect a small effect size. Only between 15.0% and 32.1% of the negative studies had adequate power to detect a medium effect size. When categorized by anatomic region, 0% to 52.6% had adequate power to detect an effect size corresponding to the minimal clinically important difference (MCID). An a priori power analysis was employed in 196 (84%) of the 233 studies. However, the power analysis could not be replicated in 46% of the studies that used a mean comparison. CONCLUSIONS: Although small improvements in orthopaedic RCTs have occurred during the past 2 decades, many RCTs are still underpowered: the sample sizes are still too small to have adequate power to detect what would be deemed clinically relevant.

Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery.

BACKGROUND: A recent study concluded that most findings reported as significant in sports medicine and arthroscopic surgery are not "robust" when evaluated with the Fragility Index (FI). A secondary analysis of data from a previous study was performed to investigate (1) the correctness of the findings, (2) the association between FI, p-value and post hoc power, (3) median power to detect a medium effect size, and (4) the implementation of sample size analysis in these randomized controlled trials (RCTs). METHODS: In addition to the 48 studies listed in the appendix accompanying the original study by Khan et al. (2017) we did a follow-up literature search and 18 additional studies were found. In total 66 studies were included in the analysis. We calculated post hoc power, p-values and confidence intervals associated with the main outcome variable. Use of a priori power analysis was recorded. The median power to detect small (h > 0.2), medium (h > 0.5), or large effect (h > 0.8) with a baseline proportion of events of 10% and 30% in each study included was calculated. Three simulation data sets were used to validate our findings. RESULTS: Inconsistencies were found in eight studies. A priori power analysis was missing in one-fourth of studies (16/66). The median power to detect a medium effect size with a baseline proportion of events of 10% and 30% was 42% and 43%, respectively. The FI was inherently associated with the achieved p-value and post hoc power. DISCUSSION: A relatively high proportion of studies had inconsistencies. The FI is a surrogate measure for p-value and post hoc power. Based on these studies, the median power in this field of research is suboptimal. There is an urgent need to investigate how well research claims in orthopedics hold in a replicated setting and the validity of research findings.

Nordic Innovative Trial to Evaluate OsteoPorotic Fractures (NITEP-group): non-operative treatment versus surgery with volar locking plate in the treatment of distal radius fracture in patients aged 65 and over - a study protocol for a prospective, randomized controlled trial.

BACKGROUND: In the literature, there are numerous studies that compare different surgical procedures in the treatment of distal radius fractures (DRF). It is, however, unknown whether operative treatment and better restoration of anatomy with volar locking plate yields a better functional outcome in the elderly population when compared with non-operative treatment. METHODS AND DESIGN: This study is a prospective, randomized, controlled, multi-center trial. The purpose will be to compare the non-operative and operative treatment of initially or early malaligned distal radius fractures in patients aged 65 and older. The primary outcome in this study will be the patient rated wrist evaluation (PRWE) score measured after 1 and 2 years. DISCUSSION: We expect that initial operative treatment of a DRF with volar locking plate will not yield superior results when compared with non-operative treatment with cast immobilization in terms of functional outcome, pain, disability, quality of life, grip strength, and number of complications. TRIAL REGISTRATION: This trial is registered on clinicaltrials.gov , identifier NCT02879656 , registration date 08/17/2016.

Rising incidence of scaphoid fracture surgery in Finland.

The aim of the present study was to investigate the incidence of scaphoid fracture surgery by obtaining data from the national discharge register in Finland, a country of 5 million inhabitants, for the period from 1997 to 2014. A total of 1380 patients with scaphoid fracture were treated surgically. Half of the patients were aged under 28 years and 84% were men. The surgical treatment of scaphoid fractures was classified into two groups. There were 640 (46%) primary fracture fixations and 740 (54%) treatments of fracture nonunion. The overall incidence of all scaphoid fixations increased twofold (from 14.8 to 30.1 per 1,000,000 person-years) and threefold in the primary fixation group (from 5.5 to 17.8 per 1,000,000 person-years) during the study period. LEVEL OF EVIDENCE: III.

Comparison of volar-flexion, ulnar-deviation and functional position cast immobilization in the non-operative treatment of distal radius fracture in elderly patients: a pragmatic randomized controlled trial study protocol.

BACKGROUND: Distal radius fractures (DRFs) are the second most common fractures, after hip fractures, seen in clinical practice. The high incidence of low-energy trauma DRFs in elderly patients raises questions about the best treatment method in terms of function, pain, and quality of life. Although the majority of these fractures are treated non-operatively with cast immobilization, valid scientific evidence of the optimal cast immobilization is lacking. In addition, several publications, including Cochrane review have outlined the need for more evidence to determine the most appropriate method of cast immobilization. METHODS: This study is a pragmatic, prospective, randomized, multi-centre trial. The trial is designed to compare two widely used cast positions (volar flexion-ulnar deviation position and functional position) for the non-operative treatment of DRF in patients over 64 years of age. The main hypothesis of the trial is that function position yields corresponding functional outcome, pain relief and quality of life when compared to the volar flexion-ulnar deviation position. The primary outcome measure is Patient Rated Wrist Evaluation (PRWE) score and the secondary outcome measures will be the Disabilities of the Arm, Shoulder and Hand (DASH) score, Visual Analogue Scale (VAS), 15-dimensional (15D) value and rate of surgical interventions. The results of the trial will be analysed after 1 and 2-years. DISCUSSION: This publication presents a prospective, pragmatic, randomized, national multi-centre trial study protocol. It provides details of patient flow, randomization, follow-up and methods of analysis of the material as well as publication plan. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT02894983 22 August 2016.